Search for: All records

Hu and Mehta (2024) posed an open problem: what is the optimal instance-dependent rate for the stochastic decision-theoretic online learning (with K actions and T rounds) under ε-differential privacy? Before, the best known upper bound and lower bound are O(logK / Δmin+logKlogT/ ε) and Ω(logK / Δmin+logK / ε) (where Δmin is the gap between the optimal and the second actions). In this paper, we partially address this open problem by having two new results. First, we provide an improved upper bound for this problem O(logK / Δmin+log2K / ε), which is T-independent and only has a log dependency in K. Second, to further understand the gap, we introduce the \textit{deterministic setting}, a weaker setting of this open problem, where the received loss vector is deterministic. At this weaker setting, a direct application of the analysis and algorithms from the original setting still leads to an extra log factor. We conduct a novel analysis which proves upper and lower bounds that match at Θ(logK / ε). 

This study tested students’ socio-cognitive outcomes in using the Open Virtual Experiment Simulator Education Tool (OVESET), a series of virtual experiment simulators designed for undergraduate polymer science education. The educational tool, covering core polymer science concepts (e.g., molecular weight distribution and polymerization kinetics), was implemented across two consecutive years in an upper-level undergraduate macromolecules course. Guided by Self-Determination Theory (SDT), this pretest–post-test study measured changes in students’ self-regulation, self-efficacy, sense of belonging, and intention to pursue a career in polymer science after using the virtual modules. In the first year, two modules were used across 3 weeks with 16 participating students; in the second year, seven modules were used over 12 weeks with 20 students. Results showed that OVESET modules significantly enhanced students’ self-efficacy in polymer science, with medium effect sizes, while changes in self-regulation, belonging, and intention to pursue a career in polymer science were not significant. This study highlights the implementation and evaluation of virtual laboratory tools in polymer science education and underscores the importance of considering student perceptions and engagement. 

This survey explores the transformative impact of foundation models (FMs) in artificial intelligence, focusing on their integration with federated learning (FL) in biomedical research. Foundation models such as ChatGPT, LLaMa, and CLIP, which are trained on vast datasets through methods including unsupervised pretraining, self-supervised learning, instructed fine-tuning, and reinforcement learning from human feedback, represent significant advancements in machine learning. These models, with their ability to generate coherent text and realistic images, are crucial for biomedical applications that require processing diverse data forms such as clinical reports, diagnostic images, and multimodal patient interactions. The incorporation of FL with these sophisticated models presents a promising strategy to harness their analytical power while safeguarding the privacy of sensitive medical data. This approach not only enhances the capabilities of FMs in medical diagnostics and personalized treatment but also addresses critical concerns about data privacy and security in healthcare. This survey reviews the current applications of FMs in federated settings, underscores the challenges, and identifies future research directions including scaling FMs, managing data diversity, and enhancing communication efficiency within FL frameworks. The objective is to encourage further research into the combined potential of FMs and FL, laying the groundwork for healthcare innovations. 

We study the implicit bias of flatness / low (loss) curvature and its effects on generalization in two-layer overparameterized ReLU networks with multivariate inputs---a problem well motivated by the minima stability and edge-of-stability phenomena in gradient-descent training. Existing work either requires interpolation or focuses only on univariate inputs. This paper presents new and somewhat surprising theoretical results for multivariate inputs. On two natural settings (1) generalization gap for flat solutions, and (2) mean-squared error (MSE) in nonparametric function estimation by stable minima, we prove upper and lower bounds, which establish that while flatness does imply generalization, the resulting rates of convergence necessarily deteriorate exponentially as the input dimension grows. This gives an exponential separation between the flat solutions compared to low-norm solutions (i.e., weight decay), which are known not to suffer from the curse of dimensionality. In particular, our minimax lower bound construction, based on a novel packing argument with boundary-localized ReLU neurons, reveals how flat solutions can exploit a kind of "neural shattering" where neurons rarely activate, but with high weight magnitudes. This leads to poor performance in high dimensions. We corroborate these theoretical findings with extensive numerical simulations. To the best of our knowledge, our analysis provides the first systematic explanation for why flat minima may fail to generalize in high dimensions. 

We propose a framework to convert (ε,δ)-approximate Differential Privacy (DP) mechanisms into (ε′,0)-pure DP mechanisms under certain conditions, a process we call ``purification.'' This algorithmic technique leverages randomized post-processing with calibrated noise to eliminate the δ parameter while achieving near-optimal privacy-utility tradeoff for pure DP. It enables a new design strategy for pure DP algorithms: first run an approximate DP algorithm with certain conditions, and then purify. This approach allows one to leverage techniques such as strong composition and propose-test-release that require δ>0 in designing pure-DP methods with δ=0. We apply this framework in various settings, including Differentially Private Empirical Risk Minimization (DP-ERM), stability-based release, and query release tasks. To the best of our knowledge, this is the first work with a statistically and computationally efficient reduction from approximate DP to pure DP. Finally, we illustrate the use of this reduction for proving lower bounds under approximate DP constraints with explicit dependence in δ, avoiding the sophisticated fingerprinting code construction. 

This paper studies the problem of differentially private empirical risk minimization (DP-ERM) for binary linear classification. We obtain an efficient (ε,δ)-DP algorithm with an empirical zero-one risk bound of Õ (1 / (γ^2εn)+|S_{out}| / γn) where n is the number of data points, Sout is an arbitrary subset of data one can remove and γ is the margin of linear separation of the remaining data points (after Sout is removed). Here, Õ (⋅) hides only logarithmic terms. In the agnostic case, we improve the existing results when the number of outliers is small. Our algorithm is highly adaptive because it does not require knowing the margin parameter γ or outlier subset Sout. We also derive a utility bound for the advanced private hyperparameter tuning algorithm.